The present invention relates to the field of machine vision applications, and more particularly to a system and method for performing machine vision analysis of an object using a reduced number of cameras.
In many machine vision applications, machine vision analysis is required of a three dimensional object, such as a rectangular prism or cube. For example, in manufacturing applications, machine vision analysis may be used to detect defects in a manufactured object by acquiring images of the object and using various types of image processing algorithms to analyze the images. As an example, a system to manufacture electrical components such as capacitors may use machine vision to examine each side of the capacitors in order to detect manufacturing defects, ensure that the capacitors are labeled, marked, or color coded properly, etc.
Machine vision applications may use image processing software operable to perform any of various types of image analysis or image processing functions or algorithms in examining an acquired image of an object. For example, pattern matching algorithms are often used, e.g., in order to compare the pattern information of the acquired image to the pattern information of a template image. Color matching algorithms may also be used, e.g., in order to compare the color information of the acquired image to the color information of a template image. Blob (Binary Large Object) analysis tools may also be used to examine an image. In various applications, pattern, color and/or Blob analysis information may be used to verify that: an object includes all necessary components in the correct locations, an object has the appropriate words, labels, or markings, an object surface is not scratched or otherwise defective, etc.
Any of type of camera or other device may be used to acquire the images to be analyzed in a machine vision application, including digital cameras, frame grabbers, line scan cameras, etc. As used herein, the term xe2x80x9ccameraxe2x80x9d is intended to encompass all such devices.
Prior art machine vision systems have required a separate image to be acquired and analyzed for each individual side of an object for which machine vision analysis is necessary. For example, in order to analyze a capacitor having six sides, a prior art system uses 6 cameras to acquire six separate images and analyze each of these images. FIG. 1 illustrates a prior art system in which a camera 10 is faced perpendicularly to a side of a cube-shaped object. Thus, the camera 10 may acquire an image including visual information of side 2 of the object but not including visual information from the other sides of the object. In order to analyze other sides of the object, additional images must be acquired. For example, FIG. 2 illustrate a prior art system including four cameras 10A, 10B, 10C, and 10D, in which the cameras are able to acquire images of sides 2, 1, 3, and 6, respectively, of the object. In order to image the top side 4 and the bottom side 5 of the object, two additional images must be acquired, which requires two additional cameras in prior art systems.
In various situations, there are several disadvantages associated with the prior art approach described above. For example, the acquisition and processing of a separate image for each object side of interest may be inefficient. Thus, it may be desirable to reduce the number of images that need to be acquired in order to examine the object, and hence reduce the number of cameras required in a machine vision system. For example, the cost of a having a separate camera dedicated to each side of the object may be prohibitive. Also, it may be desirable to position the cameras in locations other than perpendicularly facing the object sides. For example, due to the physical environment of a particular manufacturing facility, it may be necessary to position the cameras elsewhere, e.g., facing the object along a diagonal axis extending through opposite comers of the object.
One embodiment of the present invention comprises a system and method for analyzing a physical object (or a plurality of physical objects) using a reduced number of cameras and/or a reduced number of acquired images. The object may be shaped in various ways and may have any of various sizes. In one exemplary embodiment, the object is shaped like a cube or rectangular prism having six sides.
The system and method may be utilized in a machine vision application, e.g., in order to inspect manufactured objects. The method may operate to acquire an image, or a plurality of images, of the object. hi order to reduce the number of images that need to be acquired, at least one of the acquired images includes visual information for two or more sides of the object. In one embodiment, all of the acquired images include visual information for two or more sides of the object. Such an image may be acquired by a camera positioned in any of various ways such that the camera can capture visual information for multiple sides of the object.
In one embodiment, at least one of the acquired images includes visual information for three or more sides of the object. For example, with respect to an object shaped as a six-sided rectangular prism, such an image may be acquired by a camera positioned along an imaginary diagonal axis passing through the middle of the object that connects two opposite comers of the rectangular prism, wherein one comer is defined by the intersection of the three planes comprising the three sides of the rectangular prism for which visual information is acquired and the other comer is defined by the intersection of the three planes comprising the other three sides of the rectangular prism (see FIGS. 5 and 6). Such an image may also be acquired by a camera that is not positioned exactly along this axis but is positioned similarly such that the camera can acquire visual information for three sides of the object.
The acquired images of the object may then be received by a computer system and analyzed using image processing software. One embodiment of the invention comprises image processing software operable to analyze the acquired images, including the image(s) that each include visual information for multiple sides of the object. The image processing software may be operable to separate the visual information associated with each of the multiple sides of the object captured in an acquired image, e.g., to separate ROIs associated with each of the multiple sides captured in the acquired image. The image processing software may then separately analyze the visual information associated with each of the multiple sides of the object.
In one embodiment, the images may be received by a plurality of computer systems, or a plurality of image acquisition devices coupled to or comprised in a single computer system, in order to perform the analyses at a faster rate. For example, for each camera, a separate computer system connected to the camera may receive the image acquired by the camera and analyze the received images. Alternatively, for each camera, a separate image acquisition device or board coupled to the camera may capture the image, and on-board processor may analyze the captured image, or a host computer may analyze captured images from each of the image acquisition devices.
For each image that includes visual information for multiple sides of the object, analyzing the image may comprise determining regions of interest (ROIs) within the image, wherein each region of interest corresponds to one side of the object. The regions of interest within the image may be determined in various ways. For example, in one embodiment, the system is operable to acquire images such that the regions of interest have predetermined boundaries within the image. For example, the image acquisitions may be precisely coordinated with the movement and positioning of the object on an assembly line, using timing or motion control techniques. In another embodiment, the regions of interest corresponding to the object sides may be determined using image processing techniques, such as edge detection algorithms, pattern matching, color matching, Blob analysis, etc.
Once the regions of interest (ROIs) are determined, the ROIs may then be analyzed separately. Due to the angle and spatial orientation of the object with respect to a camera positioned to acquire visual information from multiple sides of the object, the ROIs may be non-rectangular regions. Thus, the image processing software may be operable to analyze non-rectangular regions of interest for defects, correct labeling, correct markings, etc.
The images or ROIs may be analyzed using any of various techniques or algorithms designed to detect any of various characteristics or defects of the object. For example, analyzing the images may comprise performing pattern matching algorithms, color matching algorithms, color and pattern matching algorithms, Blob analysis, character recognition, etc., in order to detect one or more characteristics of the object such as: physical surface defects (scratches, etc.); one or more components located correctly on the object; a correct label on the object; a correct marking on the object; correct color information on the object, etc.
The analyses performed on each image and/or performed on each region of interest (ROI) within an image may differ. For example, different sides of the object may require different types of inspection, e.g., in order to check for a correct label on one side and a set of correct components on another side.
For images including visual information from multiple sides of the object, the analyses performed on the ROIs corresponding to the sides of the object may take into account any skewing or distortion of the visual information of each side, as well as any variation in the lighting conditions of each side, which may be caused by the angle or spatial orientation of the object side with respect to the camera that acquired the image. For example, if the analysis performed for an ROI comprises performing a pattern matching algorithm, the pattern matching algorithm and/or the template image which the pattern information of the ROI is compared against may be different from what would otherwise be used if the image had been acquired from a camera faced perpendicularly to the plane comprising the object side (i.e., from a camera positioned as shown in the prior art FIGS. 1 and 2). In this case, for example, a xe2x80x9cskewedxe2x80x9d template image to be used in the pattern matching may be stored so that the pattern information of this template image will match the pattern information of the ROI, i.e., the angle and spatial orientation of the template image may match the angle and spatial orientation of the ROI. In another embodiment, the pattern matching algorithm may be altered to account for any skewing of the pattern information, e.g., by taking into account the depth of field of the camera, foreshortening of the image, etc.
In a machine vision manufacturing application, the results of the image analyses may be used to determine whether the object meets desired production standards. If separate computer systems are used to analyze the images, the results from each computer system may be considered together in making this determination. For example, another computer system may receive the results from each of these computer systems and determine whether the object meets the desired production standards, based on the results. This determination may be performed in any of various ways, as desired for a particular application. If the object does not meet the desired production standards, the object may be rejected. For example, the object may be removed from an assembly line or the system may store information indicating that the object failed the inspection. Also, images of the rejected object may be stored if desired.
One particular exemplary application is described in order to provide details regarding one specific embodiment of the system and method described above. In this example, the machine vision system analyzes manufactured capacitors having a rectangular prism shape are mounted on a lead frame carrier. Capacitor bodies project above and below the lead frame carrier, so that the lead frame carrier effectively divides the four lateral sides into two halves. Thus, the capacitor bodies effectively have ten sides: a top side, a bottom side, and eight (4*2) lateral sides. A machine vision system may be operable to inspect each of these ten sides, using a total of four cameras.
The capacitor bodies may be mounted very close together on the lead frame carrier, making it difficult for a camera to image the sides of the capacitor bodies that face each other. Thus, in order to acquire images of all the sides, the lead frame carrier may be passed over a set of sprockets causing the carrier to flex. As a pair of two consecutive capacitor bodies passes over the top of one sprocket, the gap effectively widens between the pair of capacitor bodies, allowing one camera to image the inner side of one of the capacitor bodies and allowing another camera to image the inner side of the other capacitor body. Also, one of the cameras can image the top and one of the outer sides of one of the capacitor bodies, and the other camera can image the top and the other outer side of the other capacitor body, so that each camera acquires visual information from three sides. Thus, the two cameras can together image each of the five upper sides of the capacitor bodies. Another two cameras mounted below another sprocket can image each of the five lower sides of the capacitor bodies in a similar manner.
The top side of the capacitor bodies may be marked with text. Thus, the analysis performed for the region of interest in the image that corresponds to the top side may comprise performing a pattern matching algorithm. The remaining sides of the capacitor bodies may be expected to have a uniform color and surface unless defective. Thus, simpler imaging processing functions may be used to analyze these sides. Upon detection of a faulty capacitor, the corresponding image may be stored as a file. The position of the faulty capacitor may be tracked, and the capacitor may be removed from the lead frame carrier when appropriate.